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DETAILED ACTION 

1 . This action is responsive to communication: RCE and the preliminary 
amendment filed 10/18/04 to the application filed on 05/30/97. 

2. Claims 110-121 are added. 

3. Claims 1-11, 13-16, 31-50, 63-66, 100-121 are pending in the case. Claims 1, 
10, 100, 116-121 are independent claims. 



Double Patenting 

4. A rejection based on double patenting of the "same invention" type finds its 
support in the language of 35 U.S.C. 101 which states that "whoever invents or 
discovers any new and useful process ... may obtain a patent therefor ..." (Emphasis 
added). Thus, the term "same invention," in this context, means an invention drawn to 
identical subject matter. See Miller v. Eagle Mfg. Co., 151 U.S. 186 (1894); In re 
Ockert, 245 F.2d 467, 1 14 USPQ 330 (CCPA 1957); and In re Vogel, 422 F.2d 438, 164 
USPQ 619 (CCPA 1970). 

A statutory type (35 U.S.C. 101) double patenting rejection can be overcome by 
canceling or amending the conflicting claims so they are no longer coextensive in 
scope. The filing of a terminal disclaimer cannot overcome a double patenting rejection 
based upon 35 U.S.C. 101. 

5. Claims 119 and 121 are objected to under 37 CFR 1 .75 as being a substantial 
duplicate of claims 116 and 118, respectively. When two claims in an application are 
duplicates or else are so close in content that they both cover the same thing, despite a 
slight difference in wording, it is proper after allowing one claim to object to the other as 
being a substantial duplicate of the allowed claim. See MPEP § 706.03(k). 
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Claim Rejections - 35 USC §112 

6. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

7. Claims 1-11, 13-16, 31-50, 63-66, 100-115 are rejected under 35 U.S.C. 112, 
first paragraph, as failing to comply with the enablement requirement. The claim(s) 
contains subject matter which was not described in the specification in such a way as to 
enable one skilled in the art to which it pertains, or with which it is most nearly 
connected, to make and/or use the invention. 

Independent claims 1,10, and 100, each recites among the other things, "the temporal 
order defined by the choreography information is maintained independent of the 
bandwidth of a communication channel used to send the multimedia document." 
However, the specification of the present invention discloses that: 

"Some classes of objects, however, are not interleaved with other objects in the 
multiplex section 516, regardless of the author's choreography designation. The first class 
includes those objects can not be progressively rendered. MIDI files and most standard types 
of audio and video files are examples of this class of objects ... The second class of non- 
interleaved objects includes files that are designed to be played back progressively, but that are 
authored for a particular bandwidth. These objects are referred to as time-based or 
"temporal" fil s . Examples of such temporal files are certain audio and slide show files that 
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cannot be interleaved with other data without slowing the delivery of their own temporal data 
and risking starving their players of data" (specification, page 18, lines 15-25). 

According to the specification, it appears that the temporal order is dependent of the 
bandwidth, not independent of the bandwidth used to send the multimedia document as 
claimed since the temporal file is designed or "authored for a particular bandwidth" . 

Dependent claims 2-9, 11-16, 31-50, 63-66, 101-1 15 are rejected for fully incorporating 
the deficiencies of their base claim. 

Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

9. Claims 1-3, 7-11, 14, 63-66, 100-109 remain rejected under 35 U.S.C. 103(a) as 
being unpatentable over Near et al. (US Pat No. 5,995,091 , 1 1/30/99, filed 5/10/96). 

Regarding independent claim 1 , Near discloses: 

- receiving a stream including a file that integrates media content with 

choreography information within each of at least two objects of the file, each of 
the objects including media content data and choreography information 
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associated therewith, the choreography information comprising data indicating an 
author-designated relationship between the objects of the file that defines an 
author-designated temporal order of presentation between the objects (figure 2, 
col 3, lines 28-47, col 9, lines 51-67: the interleaved playback data stream 
includes a plurality of media portions such as video chunk 1 , audio chunk 1 , 
video chunk 2, video chunk 3 where each portion has associated timestamp, 
engine destination, and message; each portion also has a simulation pointer for 
linking and arranging media portions in the stream, this shows the choreography 
information associated with media portion and the temporal order of display 
among the objects; the playback data stream indicates that the stream is 
received for playback; the authoring tool used by an author for specifying images 
and sounds for playback at specified times indicates an author-designated 
temporal order of presentation of the multimedia elements) 
- before all objects of the file are received, beginning to render media content 
encapsulated within the file based on the choreography information associated 
with objects received so as to enable display of the objects received based on 
the temporal order defined by the choreography information, wherein the 
temporal order is maintained independent of a recipient or a web server (col 3, 
line 64 to col 4, line 27, col 7, lines 50-64: the fact that the playback of the 
images and sound data stream with specified times is generated by an author for 
a selected bandwidth shows that the temporal order is defined by the author, and 
thus is maintained independent of a recipient) 
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Near does not explicitly disclose that the temporal order is maintained independent of a 
bandwidth of a communication channel used to send the multimedia document. 
However, it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made to have modified Near to include the independence of the temporal 
order over a bandwidth of a communication channel used to send the multimedia 
document for the following reason. The fact that Near provides generating a bandwidth- 
controlled presentation data stream, thereby controlling network load and providing 
predictable performance on a variety of playback systems (col 2, lines 61-67) suggests 
that the playback of the data stream in Near is not dependent on the playback systems 
and is not dependent on a bandwidth of a communication channel used to send the 
multimedia document since the network load while sending media data is controlled via 
the authoring tool. 

Regarding claims 2 and 3, which is dependent on claim 1 , Near discloses: 

- changing one object in the data file (col 10, lines 1-27: update the multimedia 
elements in the playback data stream) 

- adding an object to the data file (col 10, lines 1-27: "after processing the updated 
multimedia elements into portions ... the existing stream portions is replaced with 
the newly updated stream portions... ") 

Regarding claim 7, which is dependent on claim 1, Near discloses that each object has 
an address indicating a player that plays the object (col 8, lines 1-13; col 4, lines 47- 
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61 'identifying images and sounds to be reproduced during the playback implies 
identifying the address of the media element and the software that plays the media). 
Regarding claim 8, which is dependent on claim 1, Near discloses compressing 
information in each object (col 8, lines 13-29, col 9, lines 20-39; col 4, lines 47-61). 

Regarding claim 9, which is dependent on claim 1 , Near discloses creating an object in 
the file (col 9, lines 20-50) and locating player data within an object defining a player 
that plays the object (col 10, lines 40-67: playback media software used to identify 
individual portion of the playback data stream and the manager software calls a function 
for a player element of media data). 

Near does not disclose the created object is an unknown object. However, it would 
have been obvious to one of ordinary skill in the art at the time of the invention was 
made to have modified Near to include the created object being an unknown object for 
the following reason. Near defines the multimedia data for the playback data stream 
where each multimedia element is considered as an object in the data stream (figure 2, 
col 51-67). Thus no matter the object is known or unknown, the system always locates 
the player associated with the multimedia element. 

Independent claim 10 is for a computer system of the method claim 1 , and is rejected 
under the same rationale. 
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Regarding claim 1 1 , which is dependent on claim 10, Near discloses that at least one 
object comprises one of a textual file format, an image file format, and a sound file 
format (figure 2, col 9, lines 50-67: at least one media element comprises one of audio 
format). 

Regarding claim 14, which is dependent on claim 10, Near discloses that each object is 
a generic element of the hierarchical data file structure, such that any combination of 
objects can be grouped together to form a part of the multimedia document (figure 2, col 
9, lines 50-67: each multimedia element is a generic element such as video element or 
audio element and the combination of a video element and an audio element can be 
grouped together under a topic to form a part of the multimedia document). 
Regarding claims 63 and 65, which are dependent on claims 1 and 10 respectively, 
Near discloses that the ordered display is independent of a recipient software program 
used to render the objects (col 3, line 64 to col 4, line 27, col 7, lines 50-64: the 
playback of the images and sound data stream with specified times is generated by an 
author for a selected bandwidth, and thus the temporal order defined by the author, is 
maintained independent of a recipient software). 

Regarding claims 64 and 66, which are dependent on claims 63 and 65 respectively, as 
mentioned in claims 63 and 65 above, the ordered display is independent of the 
browser used to render the objects so that the display is presented as defined by the 
document author. 
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Regarding independent claim 100, Near discloses: 

- generating a single file that integrates the media content with the choreography 
information, wherein generating the single file comprises encapsulating within the 
single file at least two objects, each object including media content data and 
choreography information associated therewith, the choreography information 
comprising data defining a temporal order of presentation between the objects 
(figure 2, col 3, lines 28-47, col 9, lines 51-67: the interleaved playback data 
stream includes a plurality of portions such as video chunk 1 , audio chunk 1 , 
video chunk 2, video chunk 3 where each portion of media has associated 
timestamp, engine destination, message, and simulation pointer for linking and 
arranging media portions in the stream shows the choreography information 
associated with media portion and the temporal order of display between the 
objects, and where the playback data stream indicates that the stream is 
received for playback; the authoring tool used by an author for specifying images 
and sounds for playback at specified times indicates an author-designated 
temporal order of presentation between media objects) 

- before all objects of the files are received by a user, enabling the user to begin 
rendering the media content encapsulated within the file according to the 
choreography information associated with objects received, wherein the temporal 
order is maintained independent of a recipient or a web server (col 3, line 64 to 
col 4, line 27, col 7, lines 50-64: the playback of the images and sound data 
stream with specified times is generated by an author for a selected bandwidth, 
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and thus the temporal order defined by the author, is maintained independent of 
a recipient) 
Near does not explicitly disclose: 

- receiving specification of media content by a user 

- receiving designation by the user of choreography information that indicates at 
least an intended order of presentation for the specified media content 

Instead, Near provides the media content of the playback media stream, the order of 
presentation of the playback media stream defined by an author where the pointer data 
in each multimedia element for arranging the order of playing of the media element in 
the stream indicates the choreography information of each element (figure 2, col 9, lines 
50-67). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Near to include receiving specification content and 
receiving designation of choreography information for the following reason. The fact 
that the order of presentation of the media playback is defined by an author shows the 
designation of the choreography information and the presentation order of the playback. 
Further, defining the media playback suggests including a description of the media 
contents of the playback, which is equivalent to the specification of the media content of 
the playback. And it was obvious that the description as well as the order of 
presentation are received at the destination where the playback is performed. 
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Regarding claim 101, which is dependent on claim 100, Near discloses that the 
temporal order is independent of a recipient input (col 3, line 64 to col 4, line 27, col 7, 
lines 50-64: the fact that the playback of the images and sound data stream with 
specified times is generated by an author for a selected bandwidth shows that the 
temporal order is defined by the author, and thus is maintained independent of a 
recipient). 

Regarding claim 102, which is dependent on claim 100, Near discloses that the 
temporal order is independent of a recipient hardware configuration (col 3, lines 48-55: 
since the temporal order is defined by an author, the author controls the playback, and 
thus independent of the recipient hardware configuration). 

Regarding claim 103, which is dependent on claim 100, Near discloses that the 
temporal order is independent of a recipient software configuration (col 3, lines 48-55: 
since the temporal order is defined by an author, the author controls the playback, and 
thus independent of the recipient software configuration). 

Regarding claim 104, which is dependent on claim 1, Near discloses that the temporal 
order is independent of a recipient input (col 3, line 64 to col 4, line 27, col 7, lines 50- 
64: the fact that the playback of the images and sound data stream with specified times 
is generated by an author for a selected bandwidth shows that the temporal order is 
defined by the author, and thus is maintained independent of a recipient). 
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Regarding claim 105, which is dependent on claim 1 , Near discloses that the temporal 
order is independent of a recipient hardware configuration (col 3, lines 48-55: since the 
temporal order is defined by an author, the author controls the playback, and thus 
independent of the recipient hardware configuration). 

Regarding claim 106, which is dependent on claim 1 , Near discloses that the temporal 
order is independent of a recipient software configuration (col 3, lines 48-55: since the 
temporal order is defined by an author, the author controls the playback, and thus 
independent of the recipient software configuration). 

Regarding claim 107, which is dependent on claim 10, Near discloses that the temporal 
order is independent of a recipient input (col 3, line 64 to col 4, line 27, col 7, lines 50- 
64: the fact that the playback of the images and sound data stream with specified times 
is generated by an author for a selected bandwidth shows that the temporal order is 
defined by the author, and thus is maintained independent of a recipient). 

Regarding claim 108, which is dependent on claim 10, Near discloses that the temporal 
order is independent of a recipient hardware configuration (col 3, lines 48-55: since the 
temporal order is defined by an author, the author controls the playback, and thus 
independent of the recipient hardware configuration). 



Application/Control Number: 08/866,857 Page 13 

Art Unit: 2178 

Regarding claim 109, which is dependent on claim 10, Near discloses that the temporal 
order is independent of a recipient software configuration (col 3, lines 48-55: since the 
temporal order is defined by an author, the author controls the playback, and thus 
independent of the recipient software configuration). 

10. Claims 4, 13, 15, 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Near as applied to claims 1 and 10 above, and further in view of Shaw et al., 
Microsoft Office 6-in-1 3 Que Corporation 1994, pages 379-380, 384-389, 396-402, 419- 
425, 492-496. 

Regarding claim 4, which is dependent on claim 1 , Near does not disclose: 

- creating an exclusionary area within the window 

- locating an object within the exclusionary area, the object being selected from a 
group of objects including a framed image, a slide show, framed text, sound data, 
a separator, or a hyperlink 

Shaw discloses: 

- creating an exclusionary area within the window (page 401, figure 4.4) 

- locating an object within the exclusionary area, the object being selected from a group 
of objects including a framed image, a slide show, framed text, sound data, a separator, 
or a hyperlink (page 401 , figure 4.4: the data in the area within the window can be text 
and graphics). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Shaw into Robotham since Shaw teaches the window 
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features for locating objects selected from a group of specific objects providing the 
advantage to include in Robotham to enhance the use of objects from internet instead 
of merely video or audio from the media database. 

Regarding claim 13, which is dependent on claim 10, Near does not disclose that two or 
more objects have at least one common attribute, including at least one of a command 
for perception of the object, an ability to pass and receive a message, and an ability to 
supply and retrieve the data embodied in the object. 

Shaw discloses that two or more objects have at least one common attribute, including 
at least one of a command for perception of the object, an ability to pass and receive a 
message, and an ability to supply and retrieve the data embodied in the object (page 
495: since the display of the slides can be set in a temporal order by the document 
author, the slides as in the slide stream has the ability to pass and receive a message to 
automatically advance to the next slide to display the data in the slide). 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Shaw into Near since Shaw teaches the ability of passing 
and receiving a message between the objects to supply and retrieve data embodied in 
the object providing the advantage of enhancing the relationship among the media 
objects choreographed in the image stream as in Near. 

Regarding claims 15 and 16, which are dependent on claim 10, Near does not disclose 
the document forms a code segment that receives image information, and wherein the 
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image information is used to construct an image frame for a framed image that is part of 
the multimedia document. 

Shaw discloses that the document forms a code segment that receives image 
information, and wherein the image information is used to construct an image frame for 
a framed image that is part of the multimedia document (pages 400-401). 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Shaw into Near since Shaw a code segment to control the 
receiving of image information as well as the structure of the media stream providing the 
advantage of utilizing such control to edit the media stream in Near as desired. 

1 1 . Claims 5-6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Near 
as applied to claim 1 above, and further in view of Johnson (US Pat No. 5,892,847, 
4/6/99, filed 4/22/96). 

Regarding claims 5 and 6, which are dependent on claims 1 and 5 respectively, Near 
does not disclose defining as well as locating the update splash image within the data 
file. 

Johnson discloses: 

- splash image data defining a splash image and locating the splash image data within 
the data file for displaying the splash image on the computer display (col 4, lines 30-50) 

- further updating the splash image to be displayed (col 4, lines 30-63) 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Johnson into Near since Johnson teaches the process of 
displaying of a splash image, which is an element of a multimedia document providing 
the advantage of including a splash image into the image file to enhance the image 
characteristics in Near. 

12. Claims 31-50 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Near as applied to claims 1 and 10 above, and further in view of Caire et al. (US Pat No. 
5,663,962, 9/2/97, filed 9/1 5/95). 

Regarding claim 31, which is dependent on claim 1, and claims 32-34, which are 
dependent on claim 31, Near discloses a header, timestamps, and associated data for a 
data stream (col 5, line 64 to col 6, line 16). Near does not disclose that the 
choreography information further comprises an object archive for storing information 
about one or more objects, the object archive including information about the 
relationship of the object file with the document, and a multiplex section including data 
for the objects in the document. 
Caire discloses: 

- a header (col 1 , lines 65 to col 2, lines 1-2, each packet in the overall stream 
includes a header) 

- an object archive for storing information about the plurality of object files, the 
object archive including information about the level of each object file with the 
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hierarchy (col 1 , lines 65 to col 2, lines 1-2, each packet of the multimedia stream 
stores information; col 1 , lines 37-52, it is desired for instance to insert into the 
complete stream also some subtitles to be displayed during the presentation....) 

- a multiplex section including data for each of the object files of the document (col 
1, lines 65 to col 2, lines 1-9, 45-59) 

- the object files in the multiplex section are each played by a player as the 
multiplex object file is received by a receiver (col 1 , lines 65 to col 2, lines 1-2) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Caire into Near since Caire provides the choreography 
and the multiplexing features for a multimedia presentation. The combination of Caire 
into Near would provide more detailed options in the relationship of the objects in a 
multimedia document to effectively controlling and changing the presentation of the 
objects. 

Regarding claims 35, 36-39, which are dependent on claims 31 and 35 respectively, 
Near does not disclose an object number counter indicating the number of objects, a 
plurality of object descriptions, each object description describing a corresponding one 
of the objects, and a choreography group providing information about a first group of 
objects, a group object counter indicating the number objects in the choreography 
group, size and type data for each object, header data, data slices of the objects 
interleaved together, and placing one or more slice size data blocks before one or more 
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of the interleaved data slices, each slice size data block corresponding to a data slice 
and providing a size of the corresponding data slice. 
Caire discloses: 

- an object number counter indicating the number of object files (col 2, lines 10-20) 

- a plurality of object descriptions, each object description describing a 
corresponding one of the object files (col 1, lines 65 to col 2, lines 1-2, the 
header includes information of the type of a packet in the multimedia stream) 

- a choreography group providing information about a first group of object files (col 
1, lines 65 to col 2, lines 1-2, packets of different types are included in the overall 
stream as a sequence of intervals wherein the type of a packet is disclosed in the 
heading are considered as a choreography group providing information about the 
object files) 

- size and type data for each object file (col 1 , lines 65 to col 2, lines 1-2, data type 
of each packet in the multimedia stream) 

- header data (col 1 , lines 65 to col 2, lines 1-2, each packet includes a header) 

- the data slices of the object files interleaved together (col 1 , lines 65 to col 2, 
lines 1-2, the overall stream is structured as a sequence of intervals called 
packets, each of which contains data of single type, indicated in a header of the 
packet itself; since data of different types are arranged in the sequence of 
intervals called packets, the packets which are equivalent to the object files, are 
interleaved together) 
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- a first player pointer including an address of a player that plays the choreography 
group (col 2, lines 3-9, for each interval, the multiplexer has to decide from which 
the input stream it should take the data in order to construct the packets; this 
implies that the multiplexer has to decide where to point to play the overall 
stream which is equivalent to the choreography group as mentioned above) 

- locating a plurality of slice size data blocks before the interleaved data slices, 
each slice size data block corresponding to one of the data slices and providing a 
size of the corresponding data slice (col 4, lines 45-53, the number of data bytes 
and the number of header bytes in each packet show the size of each packet 
which is equivalent to the data block) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Caire into Near since Caire provides the choreography 
and the multiplexing features for a multimedia presentation. The combination of Caire 
and Near would provide the relationship of the objects in a multimedia document for 
effectively controlling and changing the presentation of the objects. 

Regarding claim 40, which is dependent on claim 31 , Near does not disclose a non- 
multiplex section following the multiplex section where the non-multiplex section 
includes one or more separate objects that are not played by a player as the separate 
object files are received by a receiver. Caire discloses a plurality of separate object files 
that are not played by a player as the separate object files are received by a receiver 
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(col 1 , lines 37-45, ...video and audio information have to be separated again, by an 
inverse of demultiplexing process, as presentation occurs on different devices...). 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Caire into Near since Caire provides the choreography 
and the multiplexing features for a multimedia presentation. The combination of Caire 
and Near would provide the relationship of the objects in a multimedia document for 
effectively controlling and changing the presentation of the objects. 

Claims 41-50 are for a computer system of the method claims 31-40, and are rejected 
under the same rationale. 

13. Claims 110-115, 118, 120-121 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Near et al. (US Pat No. 5,995,091 , 1 1/30/99, filed 5/10/96). 

Regarding claim 110, which is dependent on claim 1 , Near discloses that the media 
content is rendered independent of an author-specified bandwidth to be used to send 
the multimedia document (col 8, lines 1-28: no matter the bandwidth selected by the 
author used to send the multimedia document is sufficient or insufficient, rendering of 
the multimedia document is still performed via the playback). 
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Regarding claim 111, which is dependent on claim 1 , Near discloses that the media 
content is rendered independent of the bandwidth of the communications channel (col 
8, lines 1-28: no matter the bandwidth selected by the author used to send the 
multimedia document is sufficient or insufficient, rendering of the multimedia document 
is still performed via the playback where the bandwidth for sending the multimedia 
document is also considered the bandwidth of the communication channel since 
sending documents from one place to another is one way of communicating). 

Regarding claim 112, which is dependent on claim 10, Near discloses that the media 
content is rendered independent of an author-specified bandwidth to be used to send 
the multimedia document (col 8, lines 1-28: no matter the bandwidth selected by the 
author used to send the multimedia document is sufficient or insufficient, rendering of 
the multimedia document is still performed via the playback). 

Regarding claim 113, which is dependent on claim 10, Near discloses that the media 
content is rendered independent of the bandwidth of the communications channel (col 
8, lines 1-28: no matter the bandwidth selected by the author used to send the 
multimedia document is sufficient or insufficient, rendering of the multimedia document 
is still performed via the playback where the bandwidth for sending the multimedia 
document is also considered the bandwidth of the communication channel since 
sending documents from one place to another is one way of communicating). 
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Regarding claim 1 14, which is dependent on claim 100, Near discloses that the media 
content is rendered independent of an author-specified bandwidth to be used to send 
the multimedia document (col 8, lines 1-12: no matter the bandwidth selected by the 
author used to send the multimedia document is sufficient or insufficient, rendering of 
the multimedia document is still performed via the playback). 

Regarding claim 115, which is dependent on claim 100, Near discloses that the media 
content is rendered independent of the bandwidth of the communications channel (col 
8, lines 1-12: no matter the bandwidth selected by the author used to send the 
multimedia document is sufficient or insufficient, rendering of the multimedia document 
is still performed via the playback where the bandwidth for sending the multimedia 
document is also considered the bandwidth of the communication channel since 
sending documents from one place to another is one way of communicating). 

Regarding independent claim 118, Near discloses: 

- generating a single file that integrates the media content with the choreography 
information, wherein generating the single file comprises encapsulating within the 
single file at least two objects, each object including media content data and 
choreography information associated therewith, the choreography information 
comprising data defining a temporal order of presentation between the objects 
(figur 2, col 3, lines 28-47, col 9, lines 51-67: the interleaved playback data 
stream includes a plurality of portions such as video chunk 1 , audio chunk 1 , 
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video chunk 2, video chunk 3 where each portion of media has associated 
timestamp, engine destination, message, and simulation pointer for linking and 
arranging media portions in the stream shows the choreography information 
associated with media portion and the temporal order of display between the 
objects, and where the playback data stream indicates that the stream is 
received for playback; the authoring tool used by an author for specifying images 
and sounds for playback at specified times indicates an author-designated 
temporal order of presentation between media objects) 

- before all objects of the files are received by a user, enabling the user to begin 
rendering the media content encapsulated within the file according to the 
choreography information associated with objects received, wherein the temporal 
order is maintained independent of a recipient or a web server (col 3, line 64 to 
col 4, line 27, col 7, lines 50-64: the playback of the images and sound data 
stream with specified times is generated by an author for a selected bandwidth, 
and thus the temporal order defined by the author, is maintained independent of 
a recipient) 

Near does not explicitly disclose: 

- receiving specification of media content by a user 

- receiving designation by the user of choreography information that indicates at 
least an intended order of presentation for the specified media content 

Instead, Near provides the media content of the playback media stream, the order of 
presentation of the playback media stream defined by an author where the pointer data 
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in each multimedia element for arranging the order of playing of the media element in 
the stream indicates the choreography information of each element (figure 2, col 9, lines 
50-67). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Near to include receiving specification content and 
receiving designation of choreography information for the following reason. The fact 
that the order of presentation of the media playback is defined by an author shows the 
designation of the choreography information and the presentation order of the playback. 
Further, defining the media playback suggests including a description of the media 
contents of the playback, which is equivalent to the specification of the media content of 
the playback. And it was obvious that the description as well as the order of 
presentation are received at the destination where the playback is performed. 

Independent claim 121 includes the same limitations of claim 118, and is rejected under 
the same rationale. 

Claim 120 for a computer system of method claims 118 and 121 , and are rejected under 
the same rationale. 

Claim Rejections - 35 USC § 102 

14. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
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A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

1 5. Claims 116-117, 1 1 9 are rejected under 35 U.S.C. 1 02(e) as being anticipated by 
Near et al. (US Pat No. 5,995,091, 1 1/30/99, filed 5/10/96). 

Regarding independent claim 116, Near discloses: 

- receiving a stream including a file that integrates media content with 

choreography information within each of at least two objects of the file, each of 
the objects including media content data and choreography information 
associated therewith, the choreography information comprising data indicating an 
author-designated relationship between the objects of the file that defines an 
author-designated temporal order of presentation between the objects (figure 2, 
cot 3, lines 28-47, col 9, lines 51-67: the interleaved playback data stream 
includes a plurality of media portions such as video chunk 1 , audio chunk 1 , 
video chunk 2, video chunk 3 where each portion has associated timestamp, 
engine destination, and message; each portion also has a simulation pointer for 
linking and arranging media portions in the stream, this shows the choreography 
information associated with media portion and the temporal order of display 
among the objects; the playback data stream indicates that the stream is 
received for playback; the authoring tool used by an author for specifying images 



Application/Control Number: 08/866,857 Page 26 

Art Unit: 2178 

and sounds for playback at specified times indicates an author-designated 
temporal order of presentation of the multimedia elements) 

- before all objects of the file are received, beginning to render media content 
encapsulated within the file based on the choreography information associated 
with objects received so as to enable display of the objects received based on 
the temporal order defined by the choreography information, wherein the 
temporal order is maintained independent of a recipient or a web server (col 3, 
line 64 to col 4, line 27, col 7, lines 50-64: the fact that the playback of the 
images and sound data stream with specified times is generated by an author for 
a selected bandwidth shows that the temporal order is defined by the author, and 
thus is maintained independent of a recipient) 

- wherein the temporal order is maintained independent of a bandwidth of a 
communication channel used to send the multimedia document (col 8, lines 1- 
28: no matter the bandwidth selected by the author used to send the multimedia 
document is sufficient or insufficient, rendering of the multimedia document with 
the temporal order for the document is still performed via the playback where the 
bandwidth for sending the multimedia document is also considered the 
bandwidth of the communication channel since sending documents from one 
place to another is one way of communicating) 

Independent claim 119 includes the same limitations of claim 116, and is rejected under 
the same rationale. 
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Claim 1 17 is for a computer system of method claims 116 and 119, and is rejected 
under the same rationale. 



Response to Arguments 

16. Applicant's arguments filed 10/18/04 have been fully considered but they are not 
persuasive. 

Applicants argue that there is no motivation or teaching to modify Near to maintain 
temporal order independent of a bandwidth of a communication channel (Remarks, 
page 16). 

Examiner respectfully disagrees. 

Near provides generating a bandwidth-controlled presentation data stream, thereby 
controlling network load and providing predictable performance on a variety of playback 
systems where the presentation data stream can vary as author's desire by inserting 
data portions, images, sounds, timestamps, or commands (col 2, line 61 to col 3, line 
25). This suggests that the playback of the data stream in Near be independent of the 
playback system and be independent of the playback of the communication channel 
used to send the multimedia document since the network load while sending the media 
document is controlled via the authoring tool and the performance of the playback is 
predictable no matter what bandwidth is selected . The combination of maintaining the 
temporal order independently from the bandwidth of a communication channel into Near 
would provide an effective way to generate a presentation data stream as desired 
where the authoring tool permits interleaving the specified images and/or data for the 
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playback at the specified times at a specified bandwidth and permits the predictability of 
the playback performance at any selected bandwidth. 

Applicants argue that Near teaches away from a method or system in which "the 
temporal order is maintained independent of a bandwidth of a communications channel" 
since the authoring tool in Near requires the author to select a bandwidth for the 
playback and when the bandwidth selected by the author is insufficient to allow 
playback of the images and/or sounds at the specified times, Near "permits the author 
to abort the presentation, so that the presentation will not be played back at all, let alone 
in the correct temporal order" (Remarks, page 17). 
Examiner respectfully disagrees. 

Near discloses that in the case where the bandwidth selected by the author is 
insufficient to allow playback of the images and sounds at the specified times "the 
authoring tool element finds the delayed time at which playback can occur and notifies 
the author of the same (steps 117, 119). The author is then allowed to choose whether 
to continue and have the specified images data and/or sounds data interleaved for 
playback at the calculated delayed time (steps 121, 123), or to exit, in step 125, the 
presentation script-checking operations, in order to perform editing of the presentation 
script (col 8, lines 1-12). 0 

Clearly, when the bandwidth selected by the author is insufficient to allow playback, the 
author can choose whether to continue and have the specified images and/or sounds 
data interleaved for playback or to exit the presentation script-checking operations , in 
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order to perform editing of the presentation script, not to abort the presentation as 
argued . The presentation, therefore, still can be performed with the correct temporal 
order. 

Conclusion 

17. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

David et al. (US Pat No. 5,969,716, 10/19/99, filed 8/6/96). 
Kim (US Pat No. 5,742,238, 4/21/98, filed 4/15/97). 
Escobar et al. (US Pat No. 5,659,793, 8/19/97, filed 12/22/94). 
Conway (US Pat No. 5,745,782, 4/28/98, filed 7/3/96). 
Hubbell et al. (US Pat No. 5,966,121, 10/12/99, filed 10/12/95). 
Greenwood et al. (US Pat No. 6,640,044 B2, 10/28/03, filed 4/9/97). 
Taguchi (US Pat No. 6,546,558 B1, 4/8/03, filed 11/13/96). 
Foreman et al. (US Pat No. 6,628,303 B1, 9/30/03, filed 7/29/96). 
Gutfreund et al. (US Pat No. 6,665,835 B1, 12/16/03, filed 12/23/97). 

18. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Cong-Lac Huynh whose telephone number is 571-272- 
4125. The examiner can normally be reached on Mon-Fri (8:30-6:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Stephen Hong can be reached on 571-272-4124. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-4125. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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